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@ Communication system. 

(St) a communication system comprising at lea^ 
one mobile handheld telephone handset (1) 
adapts to communicate via a wireless tele- 
phony medium (3) with a telephone network 
handling system (3,4). ^ 

The handset comprises input devices 
(10,8,11-13) to receive input from a user and 
produce signals dependent thereupon, an 
onboard processor (7) to adapt speech input to 
produce a voice transmission signal as part of a 
telephone conversation with a third party; and 
an antenna (9) to transmit the voice transmis- 
sion signal via the wireless telephony medium 
(3). 

The telephone network handling system (3.4) 
comprises a receiver to receive the voice trans- 
mission signal, and means to forward the votee 
signal to a third party (29,30). 

The handset further comprises a first pro- 
cessor to carry out a first processing step on 
selected input signals and produce data depen- 
dent thereupon which preserves predetermined 
information necessary to cany out a remote 
second processing step, an onboard processor 
to adapt the date according to a conventional 
wireless telephony pmtocd to produce a trans- 
misston signal, and an antenna (9) to transmit 

3 the transmission signal via the wireless tele- 
phony medium to the telephone network hand- 
ling system. 

M The system further ccmiprises a remote pro- 

lyl cessor (35,36,51) adapted to receive and adapt 
_ the transmisston signal from the telephone net- 
work handling system to regenerate the date, 
iO and to carry out a second processing step on 
O the data and produce an output (43) dependent 
thereupon. 
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The present invention relates to a conrununlcation 
system. 

Systems such as Personal Digital Assistants, or 
PDAs are known which provide a variety of applica- 
tions tnduding a diary, address t>ook and computer 
applications such as wordprocessing and spread- 
sheets. PDAs are effectively "shrunken computers" 
ie. they aim to provide the functionality of a desktop 
PC in a handheld device by using miniaturised conv 
ponents. Recent attempts have also been made to in- 
corporate communication features into a convention- 
al PDA, by adding mobile phone and/or fax capabili- 
ties. 

However, since a conventional PDA contains all 
systems and applications software/hardware on- 
board, the scope of the device Is severely restricted 
due to size, processing capability and power supply 

limitations. 

In accordance with a first aspect of the present in- 
vention, we provide a communication system conn- 
prising at least one mobile handheld telephone hand- 
set adapted to communicate via a wireless telephony 
medium with a telephone network handling system; 

wherein the handset comprises means to re- 
ceive input from a user and produce signals depend- 
ent thereupon, means to adapt speech input to pro- 
duce a voice transmission signal as part of a tele- 
phone conversatfon with a third party; and means to 
transmit the voice transmission signal via the wire- 
less telephony medium; and 

wherein the telephone network handling sys- 
tem comprises means to receive the voice transmis^ 
sk>n signal, and means to forward the voice signal to 
a third party; 

characterised in that the handset further com- 
prises first processing means adapted to carry out a 
first processing step on selected input signals and 
produce data dependent thereupon which preserves 
predetermined informatk>n necessary to carry out a 
remote second processing step, means to adapt the 
data according to a conventional wireless telephony 
protocol to produce a transmission signal, and means 
to transmit the transmission signal via the wireless 
telephony medium to the telephone network handling 
system; and 

in that the system further comprises means to 
receive and process the transmission signal from the 
telephone network handling system to regenerate the 
data, and second processing means positioned re- 
mote from the handset and adapted to can-y out a 
second processing step on the data, and to produce 
an output dependent thereupon. 

The system according to the present invention 
provides a means of delivering potentially unlimited 
processing power to a mobile handset, since the re- 
mote processor (which is generally fixed) is not limit- 
ed by size or power consumption requirements. 
The present invention typically includes a wire- 
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less telephone device for use in an associated wire- 
less telephony network environment (with the con- 
ventional "telephony" features of means to adapt 
speech input as part of a conventional telephone con- 

5 versation with a third party), along with the additbnal 
feature of further processing distributed between the 
handset and a remote processor which may be used 
in a variety of alternative applications. This enables 
effecth^e human to computer interaction over a wire- 

10 less telephony medium by providing means In the 
handset to carry out a first processing step which, 
whilst processing, for example compressing, the input 
signals enough to enable acceptat>ly error free trans- 
mission over the wireless telephony network, also 

15 preserves key Information which Is required to carry 
out a remote second processing step in a central 
processor, or renrwteiy running application. Typically, 
the telephone network handling system comprises a 
wide area cellular network which utilises the GSM 

20 (General Standard Mobile) protocol, (with an associ- 
ated bandwidth for data transmlsston of 9.6 KBps). Al- 
ternatively, It may comprise the DCS 1800 or DECT 
(Digital European Cordless Telephony) environ- 
ments. 

25 The telephone network handling system is typi- 

. cally a circuit switched system providing real-time, 
communication between the handset and the third 
party (in the case of a telephone conversation), and 
between the first and second processing means (in 

30 the case of data transmission). This may be contrast- 
ed with a "packet switched" or "message switched" 
comnmjntoation environment whteh can only be used 
for data transmission. 

Typically the means to process outgoing speech 

35 input for transmission and incoming voice signals for 
audio output comprises a modem, and may also fur- 
ther comprise a codec in the case of digital transmis- 
sion. 

The first and second processing steps nr^y be 

40 consecuth/e steps in a speech recognition system. 
For Instance, the first processing s>tep in the handset 
may be the first step in a speech recognition process, 
and the second processing step may comprise the re- 
maining steps in the speech recognition process, pro- 

45 viding text/instructions as the output 

Typically the first processing step in the speech 
recognition process produces an output which retains 
key features of the input speech to allow the speech 
recognition process to be completed in a remote cen- 

50 tral server, whilst also conforming with the bandwklth 
limitation of the wireless telephony medium (9.6KBbs 
in the case of GSM). 

Typically the first step in the speech recognition 
process comprises an initial feature analysis which 

55 converts the input speech signals according to a 
known algorithm (such as linear predictive coding) 
into parameter coefficients. The second processing 
step which is carried out in a remote central processor 
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typically comprises comparing the received parame- 
ter coefficients to a library of known sub-word units 
(such as phonemes, sub-phonemes or tri-phones) 
followed by further analysis against lexical, syntactic 
and semantic models to provide output 
text/instructions. 

The output from the central processor may be re- 
turned to the handset for display in the form of text, 
and may also be input as instructions to a further, re- 
motely processed application which accepts text as 
input 

The approach to voice processing recognises 
that significant resources, both in tenms of processing 
power and data storage, are required by state-of-the- 
art recognition engines. Rather than delivering a com- 
promise solution through miniaturisation (as is the 
case with PDAs), limited in both vocabulary range 
and performance, the handset supports the initial sta- 
ges of voice recognition processing, i.e. the initial fea- 
ture analysis, on the handset and piasses the results, 
now sufficiently reduced In volume to fit the restricted 
data channel bandwidth offered by mobile/cordless 
networks, to a large, powerful, multi-user recognition 
engine located centrally. 

Alternatively, the first and second processing 
steps may be separate processes in a fax transmis- 
sion system, or first and second analysing steps in a 
handwriting recognition system, similar to the speech 
recognition system. 

In the case of a handwriting recognition system, 
text may be Input to the handset via a touch sensitive 
screen and transmitted to the centra! processor using 
a standard facsimile protocol (or any other protocol 
which provides an appropriately high quality transfer 
method such that essential features in the written text 
are not lost). The central processor then carries out a 
handwriting recognition procedure on the fax data to 
interpret the handwriting and return it to the handset 
as recognised text 

In a further alternative, the first and second proc- 
essing steps may be used in a "remote desktop" ap- 
plication -in which the handset acts as a remote in- 
put/output device for a computer. The first processing 
step typically comprises converting input from the 
user (for instance cursor control instructions) into ob- 
ject-level graphics code, tailored for a particular op- 
erating system (such as MS-Windows™ , Macin- 
tosh™ or any other object-oriented graphical Inter- 
face) installed on a remote PC. The code is modulated 
according to the required wireless protocol and trans- 
mitted, via the telephone network handling system, to 
the remote PC. In this case the remote processor is 
contained in the PC, which decodes the object-level 
graphics code and inputs the instructions to a running 
application. 

Built-in intelligence in the handset Identifies the 
input activity on an object basis, and transmits only a 
coded-representation of the object manipulation to 



the remotely executing application. Taking the exam- 
pleof drawing a line using a pointing device, the hand- 
set determines the type of object being input in this 
case a line, and its properties (starting and ending co- 
5 ordinates, thickness, colour, etc.) which can then be 
transmitted in coded form to the application. The de- 
vice drivers associated with the remotely-executing 
application decode the input by reference to a stan- 
dard object library (which mirrors that held locally on 
10 the handset), and pass this into the application with 
no appreciable delay. The representation of the line 
shown on screen is generated locally by the handset 
and only altered if the remote application dictates a 
change. To the user, the result is instantaneous re- 
ts sponse. M w. 

In this case the application functionality available 
to the handset user is not limited by the capabilities 
of the on-board processor. In the "remote desktop" 
application, by communicating with and remotely con- 
20 trolling an application executing elsewhere, the hand- 
set delivers the power of that remote processor to the 
mobile user. Further, most users of the handset will 
already have a suite of applications, tailored to their 
information requirements, resident on their existing 
25 desktop PC. With the handset delivering access to all 
these resources from any tocation, there Is no need 
for additional application software on the handset 
with Its cost and support implicattons. nor is a conrv 
plex conf iguration exercise necessary in order to es- 
30 tablish connections to a range of information sources. 

As a further development of the "remote desktop" 
concept a multi-user computer, capable of running 
application software which is commonly found on a 
personal computer, is located remotely within a tele- 
35 phone network. That is. the multi-user computer pro- 
vides application execution capability to a large pop- 
ulation of remote users as and when they need it 
Handset users who are familiar with personal oomput- 
er applications, but do not have an appropriately con- 
40 figured personal computer to access, can run these 
applications within a Virtual PC" environment on the 
multi-user node. This mechanism can delh/er the ben- 
efits of personal computer applications to a handset 
user without requiring the capital outlay associated 
45 with a personal computer purchase. 

In accordance with a second aspect of the pres- 
ent invention, we provide a communication system 
comprising at least one mobile handheld telephone 
handset adapted to communicate via a wireless tel- 
50 ephony medium with a telephone networic handling 
system; 

wherein the handset comprises means to re- 
ceive input from a user and produce signals depend- 
ent thereupon, means to adapt speech input to pre- 
ss duce a voice transmission signal as part of a tele- 
phone convereation with a third party; and means to 
transmit the voice transmission signal via the wire- 
less telephony medium; and 
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wherein the telephone network handling sys- 
tem comprises means to receive the voice transmis- 
sion signal, and means to forward the voice signal to 
a third party; 

characterised in that the handset Is adapted to 
transmit and receive input/output data via the wire- 
less telephony medium and the telephone network 
handling system whereby the handset acts as an In- 
put/output device for an application program execut- 
ing on a computer positioned remote from the hand- 
set and attached to the telephone network handling 
system. 

As previously described, the computer may be a 
desktop personal computer (PC) or a computer pro- 
ykiing a parallel multi-user environment. 

Again, the telephone network handling system is 
typically a circuit switched system. The limited band- 
width capabilities of the wireless telephony medium 
are typically overcome by converting input/output into 
an object-level graphics description protocol for 
transmission. 

An example of the present invention will now be 
descrit>ed with reference to the accompanying Fig- 
ures in which:- 

Figure 1 illustrates a handset with associated 

network architecture; 

Figure 2 shows the exterior of the handset, with 

typical user input/output devices; 

Figure 3 illustrates the apparatus in a typical 

speech recognition process; 

Figure 4 is a flow diagram showing associated 

speech recognition steps; and. 

Figure 5 illustrates the apparatus in a rennote 

desktop aspect of the present invention. 

The network architecture is shown in Figure 1 , 
and comprises four key functional entities: handset 1 ; 
transport infrastructure 2 (signalling and bearer 
channels) comprising wireless 3 and fixed network 
transmission media 4; central processing, formatting 
and control platform 5; and service provision platform 
6. Item 6 is a generic entity representing assorted ser- 
vk:e provision platforms, which might include a store- 
and-forward fax server, interactive banking service or 
electronic mail access facility. 

A single processor 7 in the handset 1 is respon- 
sible for an overall control function, data processing 
and transaction functions associated with on-board 
applications, and DSP functions. The processor is 
typically a RISC based processor which offers high 
flexibility i.e. the chip power can be used for radio, 
computing applications, DSP or split between all 
these. The handset also supports a 16KHz micro- 
phone 8 with noise cancellation, and antenna 9. From 
hereon the processor 7 will be referred to as DSP 7, 
to differentiate it from other processors whteh will be 
described. 

The handset also comprises input and output de- 
vices as shown in Figure 2 for interaction with a user. 



The handset comprises LCD display screen 10 for 
displaying fax. E-mail, menu options and other visual 
output As can be seen in Figure 2, the area of the 
screen 10 occupies a significant proportbn of the 

5 area of the handset 1 which is presented to the user. 
The handset also comprises input buttons 11,12 and 
a cursor control 13, along with the telephone micro- 
phone 8 and loudspeaker 14 which are positioned for 
optima] ear-to-nrK>uth spacing. Typk^ dimensions of 

10 the handset are a length 15 of 170mm, a wkJth 16 of 
70mm and a depth {not shown) of 23mm. This allows 
the handset to be easily held in the hand. The handset 
also comprises an antenna (not shown) which is inte- 
grated into the case and does not require extension 

15 for operation. 

The handset 1 may communicate with any con- 
ventional wireless ' communication environment 3, 
such as GSM 900 or DCS 1800. Alternathrely, In a 
cordless system the mobile handset may communi- 

20 cate with a cordless (for instance DECT) base station. 
Apart from routing calls between the handset 1 and 
central platform 5, the transport Infrastructure 2 may 
also set up telephone conversations t)etween the 
handset 1 and a third party 29 or 30. either via other 

25 base stations in the wireless network 3» or via the 
PSTN 4 respectively. 

The centra] platform 5 typically comprises a pre- 
sentation manager 26. The presentation manager 26 
is responsible for the management and control of the 

30 interface between the central platform 5 and the 
handset 1, and the acceptance of input from the de- 
vice. The presentation manager 26 also formats the 
output from any of the service provision platforms, 
and interprets events which occur within the hand- 

35 set* s Interface. The central platform 5 also carries out 
a variety of functions. Including processing of Ennall 
and fax data whk:h is forwarded/received from the 
handset 1 . 

Multi-user database 32 Is used to store data re- 

40 lating to each handset user. 

Item 28 represents a generic "service proxy" ca- 
pability, which forms part of the central platform 5. 
This offers the "expected" interface to a service pro- 
vision platform 6. e.g. It emulates a VT100 terminal. 

45 Output is passed to the presentation manager 26 to 
be formatted for the hairrdset whilst events interpret- 
ed by the presentation manager 26 as requests to the 
service are appropriately translated and issued to the 
relevant platform 6. 

50 Voice processing host 35, is configured to partici- 

pate in a distributed speech recognition environment, 
and also incorporates voice synthesis capabilities to 
deliver audio output, such as prompts and instruc- 
tions, to the handset user. 

55 Handwriting recognitk>n host 36 converts hand- 

written image data from the handset into text (descri- 
bed later). 

Atypical speech recognition application will now 
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be described with reference to Figures 3 and 4. 

Figure 3 shows the apparatus involved in a proc- 
ess of speech recognition. The first step in the speech 
recognition process is earned out in DSP 7 in the 
handset, and the remaining steps are carried out in 
voice processing host 35. The host 35 can deliver tex- 
tual output in an appropriate format in a manner sim- 
ilar to the output of a keyboard. The host 35 is respon- 
sive to speech input derived from the handset 1 by 
recognising speech units generated by the handset 
(eg. phonemes, sub-phonemes etc) and converting it 
into a textual representation. 

As will be appreciated, the mobile handset 1 is re- 
mote from the host 35 and this has. in the past, 
formed a substantial barrier to the provision of a fully- 
responsive high-quality speech recognition system 
by way of the currently available telephone network. 

As will be appreciated, in order to assist the host 
35 in attending to the required fully-responsive and 
high-quality recognition process, the speech Infonra- 
tton provided to the host 35 should carry high fidelity 
Information similar to that achieved with CD or DAT 
audio systems. This requires a bandwidth In the order 
of 22 Khz and, although the host 35 is available to 
process the information provided In such a band- 
width, the communication link formed between the 
handset 1 and the speech recognition apparatus 35 
is not capable of providing such a t)andwidth. 

The Invention overcomes this problem by carry- 
ing out a unique analysis and compression process 
on the speech infonmation captured by way of the mi- 
crophone 8. The analysed and compressed data only 
requires a bandwidth of an order that is commonly of- 
fered by present networks, for example 9.6 Kbps. The 
analysed and compressed data, once receh^ed by the 
host 35. is then used as a source for reconstructed 
speech data having an appropriately wide bandwidth, 
e.g. in the order of 22 Khz, which allows for the re- 
quired high-quality recognition procedure to be car- 
ried out at the host 35. 

Figure 4 is a flow diagram illustrating the various 
steps which are canned out by the apparatus ele- 
ments in Figure 3. 

The speech infonnfiation that is to be subjected to 
the speech recognition process and subsequently to 
be employed as the control informatton for a renrwte 
system, is captured (step 60) by way of the micro- 
phone 8. having an appropriate transducer, and is fed 
to the DSP 7. The DSP 7 operates to capture and digi- 
tise the speech information (step 62) at high-fidelity 
levels with a bandwidth in the order of 22 Khz. Sub- 
sequently, initial feature analysis (step 63) of the digi- 
tised speech signal is carried out in the DSP 7. Typical 
algorithms which may be employed at this stage are 
linear predictive coding orf liter-bank analysis. The al- 
gorithm is chosen to preserve the full range of input 
frequencies (up to approximately 1 6KHz). This allows 
key features to be preserved after transmission, al- 
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lowing detailed (for instance, dictation quality) 
speech recognitton at the host 35. The output of initial 
feature analysis comprises a stream of parameter 
coefficients 64. These coefficients may be coded 

5 further by the DSP if required to comply with the 
bandwidth of the transmission medium (for instance 
by temporal decomposition interpolation in a step not 
shown in Figure 4) and are then modulated at 65 in a 
conventional way to provide a radio transmission sig- 

10 nal according to the required protocol (for instance 
GSM. DECT or DCS 1800). The transmission signal 
is then passed to antenna 9 which transmits the sig- 
nal at 66. 

The transmitted signal 67 is then received and ra- 
15 dk) demodulated in a conventional way by a base sta- 
tion 20 in the wireless telephone handling network 3 
and is eventually passed as digital data (via a number 
of units Indicated generally by dotted lines 53) to the 
host 35. Depending on which protocol is used, the 
20 transmitted data is received at a Private Branch Ex- 
change (PBX) or a Mobile Switching Centre (MSC) 
before being delivered to the main conrununication 
network. The Initial feature analysis parameter coef- 
ficients may have been regenerated in the previously 
25 described demodulation step. Alternatively, further 
decompression may be required by the host 35 to re- 
generate the initial feature analysis coefficients (if for 
example the coeff teients had been coded by temporal 
decompositk>n interpolatton). 
30 The coefficients are then used to look up a library 

41 of phonemes, sub-phonemes, diphones. tri-pho- 
nes or other sub-word units, depending on what the 
coefficients represent This is indicated at 69 in Fig- 
ure 4. The selected sub-word units are then analyzed 
35 by processor 42 at 70, which outputs text 71 which 
may be used in a number of ways. For instance, the 
textual output 43 may be retransmitted to the handset 
for display. In addition, the textual output may be used 
to provide input for a separately running application 
40 which recognises textual inpui 

Atypical speech recognition procedure has been 
outlined above, but in general any conventional 
speech recognition process may be used, with part of 
the process being carried out in the handset 1, and 
45 the rest in the central server 34. The division of tasks 
is determined to minimize loss of informatfon and to 
optimize the use of the DSP 7. 

In one specific embodiment, the step of initial 
feature analysis 63 may involve generating data that 
so is representative of speech elements, (e.g. ph<>- 
nemes. sub-phonemes etc.) of the information digi- 
tised in the DSP 7. The following description shall de- 
scribe the use of phonemes, although the speech ele- 
ments nnay be sub-phonemes or tri-phones. 
55 The data representative of the phonemes after 

encoding is arranged to be transmitted by the chosen 
protocol (e.g. DECT. GSM 900. DCS 1 800 etc.). The 
DSP 7 contains a library of a large number of pho- 

5 
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nemes which can be stored on approxinr^tely 2 M 
bytes of ROM. The encoded data signal derived from 
the DSP 7 for transmission via the chosen protocols 
Is a highly accurate representation of the complete 
speech infonmation captured by way of the micro- 
phone 8. The compression of the speech Infonmation 
thereby achieved by way of the DSP 7 Is of particular 
relevance since transfer of the encoded data requires 
a far narrower bandwidth than was Initially required to 
capture the speech data by way of the microphone 8. 
in particular, the bandwidth of the encoded data falls 
within the bandwidth that Is available for transmission 
via the chosen protocol and via the Network PSTN in 
general. 

The encoded data representation of the pho- 
nemes Is then transmitted through the network with- 
out suffering any undue degradation and is directed 
from the gateway 34 to the voice processing host 35. 
This host 35 includes a library of data relating to all 
possible phonemes and mirrors the library of the DSP 
7. 

Thus, upon receipt of the encoded data transmit- 
ted via the network from the handset 1. the host 35 
is arranged to reconstruct the speech information by 
reference to the contents of its library. The recogni- 
tion process can t>e then carried out on a relatively 
high-fidelity speech infomiation having the band- 
width of 22 Khz. 

The invention is particularly advantageous In that 
the speech recognition process eventually carried out 
in the host 35 is arranged to be speaker-specific since 
a smartcard level of user identification and security is 
inherent within the handset 1 . As such, the central 
platform 5 and associated host 35 fbnms a powerful 
"recognition engine* which can be supplied with the 
necessary input information via the telephone net- 
work from a small and relatively Inexpensive handset 
1 . Thus, the actual recognitk>n procedure can be exe- 
cuted remote from such a handset and In an appro- 
priate environment that corresponds to specialized 
test conditk>ns. Also, since the network sees the 
speech information merely as encoded data, full error 
correctk)n and encryption can be applied where ap- 
propriate. Although this will cause response times to 
exceed the generally acceptable limit for conversa- 
tional speech, such variatk>ns do not present a signif- 
icant problem for speech recognition purposes in 
which mean delays of up tof ive seconds are generally 
considered acceptable. In any case, since phrases 
can be used as input instructions for information re- 
trieval, greater delays, even of up to ten seconds, are 
regarded as reasonak>le. Primarily, a handset owner 
is provided with access to a speech recognitton capa- 
bility that would not be otherwise available from a 
handset, or from a laptop PC, arid would be particu- 
larly and prohibitively expensive in a dedicated desk- 
top machine. 

Also, by virtue of the architecture of the present 



invention, the user of the handset 1 will be able to 
benefit directly from all improvements that are made 
to the central platform 5 and the host 35 as the related 
technology advances. 

5 As illustrated in Figure 1, the central platform 5 

can provide for access of the data appearing on the 
network for other applications 6 which might require 
vocal, textual or numerical conunands such as the 
service operations of banks and retailers or other 

10 dedicated services provided on the network, for ex- 
ample the so-called Wings appllcatrons provided by 
Mercury Conrtmunications Limited. The level of secur- 
ity and user identif icat'ion inherent in the use of the 
DSP 7 as an analysing and encoding device will prove 

15 particularly useful for such service operations. 

The proviston of such high-quality speech recog- 
nition by way of a handset 10 and the PSTN provides 
the basis for a particularly advantageous multi-media 
data processing systent For example the combina- 

20 tion of voice response, speech recognition, speaker 
ldentlficatk>n, text/Image dteplay, touch sensitive 
screen etc. provkles for a system offering a wide va- 
riety of appllcattons In a particular user-friendly and, 
responsive manner. 

25 A "remote desktop" application of the present in- 

vention will now be described with reference to Figure 

Existing protocols which support detached in- 
put/output require the transmission of a considerable 

30 amount of network traffic in order to complete, for ex- 
ample, the drawing of a line using a pointing device. 
The remotely executing applicatk>n is supplied with 
input information reflecting each minute movement of 
the mouse, and responds with output instructions for 

35 each corresponding extension of the line to be dis- 
played on the screen. In practice any attempt to im- 
plement this conventional architecture on a mobile 
network would deliver considerable, unacceptable 
delay between the input action, and the resulting out- 

40 put. 

The "remote desktop" feature according to the in- 
vention allows the handset 1 to act as an Input/output 
device for a remote desktop personal computer (PC), 
Indicated generally at 50. 

45 The handset 1 sets up a call with the PC via a lo- 

cal base station 20 associated with a particular wire- 
less telephony network. The call is then forwarded to 
the PC via a telephone network handling system (In- 
dicated generally in dotted lines). 

50 As well as displaying screen information, the 

handset receives input from a user and thus acts as 
a "renrwte control" for the PC 50. This is achieved over 
a limited bandwidth communication medium by the 
use of an object-level graphics description protocol 

55 which is optimised for low bit-rate utilisation within the 
network which links the application (on the PC 50) 
and the handset 1. 

The PC comprises a processor 51 which runs an 

6 
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application programme 52 supported by the Windows 
operating system 53. The Windows operating system 
53 receives input from a keyboard 54 and keyboard 
driver 55 wtiich s typically converted into an object- 
level description protocol for input to the application 
programme 52. Similarly, output from the application 
programme 52 is converted into windows object-level 
function calls with reference to the appropriate win- 
dows libraries and output as function calls to screen 
driver 56 which drives the screen 57. The keyboard 
driver 55 and screen driver 56 are configured to input 
and output respectively Infomiation via a serial port 
58 which is connected to a network adaptor 71 con- 
nection to a telephone network. For Instance the 
adaptor 71 may be an ISDN terminal adaptor. 

The keyboard driver 55 and screen driver 56 may 
be configured by the PC manufacturer to divert in- 
put/output function calls toAfrom the serial port 58 (as 
shown in Figure 5). Alternatively, a conventional PC 
may be adapted with a card inserted into the PC to en- 
able the Interface with the remote handset In a fur- 
ther alternative, the PC 51 may be connected to a cor- 
porate server which has software which provides the 
interface between a number of PCs attached to it and 
the external telephone network. 

Turning now to the handset 1, the DSP 7 receives 
input from input means generally indicated at 70 
which may be the cursor control 13 or input from the 
touch sensitive screen 10. Built-in intelligence in the 
DSP 7 identif ies the input activity on an object basis, 
and transmits only a coded representation (In appro- 
priate Windows™ format) of the object manipulation 
to the remotely executing application. Taking the line 
drawing example outlined above, the DSP 7 would 
determine the type of object being input with refer- 
ence to a Windows™ object library. In this case the 
DSP 7 determines that the object is a line, and also 
determines its properties (starting and ending coor- 
dinates, thickness, colour ete) which can then be 
transmitted in coded form to the applicatfon. The key- 
board and screen drivers 55.56 associated with the 
remotely iexecuting application decode the input by 
reference to a standard object library (which mirrore 
that held locally on the handset) and pass this into the 
applicatton with no appreciable delay. The represen- 
tation of the line shown on screen is generated locally 
by the DSP 7, and only altered If the remote applica- 
tion 52 dictates a change. To the user the result Is in- 
stantaneous. 

When the remote application 52 dictates a 
change, screen driver 56 outputs the appropriate 
function call which is transmitted to the handset 1 via 
adaptor 71 , and telephone network Base Station Con- 
troller 20. The DSP 7 receives the signal and process- 
es it with reference to the stored standard object I ibra- 
ry; Appropriate output is then displayed on the screen 
10. 

Instead of accessing a conventional PC as shown 



in Figure 5, the handset way access a "multi-user" 
which provides applicatton execution capability to a 
large population of usera in parallel, as and when they 
need it 

5 The use of the handset as an Input/Output and 

control device for a store-and-forward fax service will 
now be described. 

A store-and-forward fax service acts as a mail- 
box for facsimile messages. A centrally connected 
10 node offere a line-terminating interface to which in- 
coming fax messages are routed e.g. via call- 
forwarding or as part of an intelligent peripheral ar- 
rangement (such as Mercury's Wings platform). 
These incoming faxes pass through the defined 
is fax interface and are stored on disk in digital form. Re- 
ceipt of the fax may trigger a notification process, 
alerting the user to the presence of new messages, 
via a pager message, for example. 

Access to stored messages Is traditionally ach- 
20 ieved via a telephone handset DTMF interface, 
through which a suitable destination fax number is In- 
put by the user. On specif toatlon of the fax number, 
the mailbox Is 'unloaded*, through converston of the 
stored digital data across the defined fax interface, 
25 and transmission in analogue form. 

The handset's large, touch-sensitive screen al- 
tovirs It to act as a terminal for the input of handwritten 
or sketched faxes, while Its scrolling display capabil- 
ity allows externally originating faxes to be read. In 
30 addition, the ability to participate in a distributed 
speech recognitton environment will support spoken 
text or command input 

The combination of the handset and a stbre-and- 
forward fax service offers an effective solution to mo- 
35 bile faxing requirements. The gateway can receive 
large faxes on the user's behalf and alert the user via 
the handsets graphical user Interface. This can indi- 
cate the sender's details (extracted from the fax 
header) along with the message size. A user can then 
40 choose to view the message on the handset's screen, 
scrolling down each page, or issue a command for the 
fax to be forwarded to a nearby fax machine if the 
message is large or a hard copy is required. 

Using the touch sensitive screen, the user can 
45 annotate an incoming fax, perhaps signing it as au- 
thorisation, for example, and return it to the sender or 
forward to another party. Re-trying busy nunibere 
and forwarding to multiple destinations are features 
supported within the store-andrforward environment. 
50 thus reducing the amount of time required to process 
and reply to messages. 

In most instances, the transfer of facsimile infor- 
mation between the handset and the service host will 
use a standard fax protocol. Group III fax is included 
55 within the GSM phase 2 standard, and is also support- 
ed within many other cellular and cordless environ- 
ments. Howevjgr, there Is an opportunity for small 
messages, such as a screen-sized "fax note", to be 
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transmitted via a proprietary protocol over a standard 
transport mechanism. As an example, Apple and Mo- 
torola have developed a compression standard which 
can represent "raw ink" (handwritten text and sketch- 
es) from an Apple Newton Message Pad screen In 
around 500 bytes, originally as part of their work on 
two-way paging around Asia. 

Given that the SMS facility for out of band mes- 
saging within GSM and DCS 1800 (and the COMS fa- 
cility within DECT) offers a throughput around 160 
bytes per message.va fax note could be sent over 4 
such messages. Using this method to move the fax 
representation from the handset to the service host 
would eliminate the expense of a sfwitched bearer 
channel, and would also allow a call in progress to 
continue simultaneously. 

A method of implementing a distributed handwrit- 
ing recognition environment will now lie described. 

Conventidnal Personal Digital Assistants have 
attempted to allow text input via handwriting recogni- 
tion. This process involves the use of local resources 
to Interpret handwritten text entered on a touch- 
sensith^e screen. Unfortunately, this has not per- 
formed particularly effectively. 

The availability of nnore powerful remote proc- 
essing resources, along with more extensive refer- 
ence data, significantly improves the perforimance of 
the handwriting recognitton process. Theirefore a dis- 
tributed system according to the invention, in which 
the handset gathers and presents data for processing 
by a centrally located host provides significant advan- 
tages. 

In a similar fashion to the voice processing sys- 
tem described prevtously, the input data needs to be 
transmitted to the central resources without losing 
any features which contribute to the accuracy of proc- 
essing. Handwriting can be recognised on the basis 
of dynamic features, and stiatic graphic features. Dy- 
namic features may be analysed at the handset by the 
DSP 7, and transmitted to a remote handwriting rec- 
ognition processor 36 along with graphic features.. 

Preferat>iy the graphic features are transmitted 
via a protocol which retains predetermined features of 
the input handwriting signals to allow the handwriting 
recognition process to be completed in the remote 
processor. 

Standard facsimile protocols offer an appro- 
priately high quality transfer method for graphic fea- 
tures, such that the handset (or any other piece of 
equipment which conforms to standard facsimile 
standards) can deliver handwriting for recognition via 
fax, and display the recognised output. 



Claims 

1 . A communication system comprising at least one 
mobile handheld telephone handset (1) adapted 
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to communicate via a wireless telephony medium 
(3) with a telephone network handling system 
(3,4); 

wherein the handset comprises means 

5 (10.8.11-13) to receive inputfrom a user and pro- 

duce signals dependent thereupon, means (7) to 
adapt speech input to producei a voice transmis- 
sion signal as part of a telephone conversation 
with a third party; and means (9) to transmit the 

10 voice transmission signal via the wireless teleph- 

ony medium (3); arid 

wherein the telephone network handling 
system (3,4) comprises means to receive the 
voice transmission signal, and means to forward 

IS the voice signal to a third party (29,30); 

characterised in that the handset further 
comprises first processing means (7) adapted to 
carry out a first processing step on selected input 
signals and produce data dependent thereupon 

20 which preserves predetermined information neo- 

essary to carry out a remote second processing 
step, means to adapt the data according to a con- 
ventional wireless telephony protocol to produce 
a transmission signal, and means (9) to transmit 

25 the transmission signal via the wireless teleph- 

ony medium to the telephone network handling 
system; and 

in that the system further comprises 
means to receive and process the transmission 

30 signal from the telephone network handling sys^ 

tem to regenerate the data, and second process- 
ing means (35,36,51) positioned remote from the 
handset and adapted to carry out a second proc- 
essing step on the data, and to produce an output 

35 (43) dependent thereupon. 

2. A system according to daim 1 wherein the first 
and second processing means are adapted to 
carry out first (63) and second (69,70) steps in a 

40 speech recognition process. 

3. A system according to claim 2 wherein the select- 
ed input signals comprise speech signals, and 
wherein the first processing means provkJes an 

45 output (64) which retains predetermined features 

of the input speech signals to enable the second 
processing means (35) to carry out the remaining 
steps of the speech relcognition process. 

50 4. A system according to claim 3, wherein the sec- 
ond processing means (35) provkles an output 
(43) comprising text signals representative of the 
textual content of the input speech signals, and 
returns the text signals to the handset for output 

55 as text 

5. A system according to claim 3 or 4 wherein the 
first processing step comprtees. initial feature 

8 
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analysis. 

6. A system according to any of the preceding 
daims. wherein the first and second processing 
means are adapted to carry out first and second s 
steps in a handwriting recognition process. 

7. A system according to daim 6 wherein the select- 
ed input signals comprise handwriting signals in- 
put via a touch sensitive screen (1 0) on the hand- to 
set and wherein the f irst processing means pro- 
vides an output which retains predetermined 
graphic features of the input handwriting signals 

to enable the second processing means (36) to 
carry out the remaining steps of the handwriting is 
recognition process. 

8. A system according to claim 7 wherein the trans- 
mission signal is modulated according to a con- 
ventional facslrnile protocol. 20 

9. A system according to daim 7 or 8 wherein the seo- 
ond processing means provides an output compris- 
ing text signals representative of the textual content 

of the input handwriting signals, and returns the text 25 
signals to the handset for output as text 

10. A system according to daim 4 or 9 wherein the 
text is Input as instructions to a further remotely 
processed application. 

11. A system according to any of the preceding 
daims, wherein the handset acts as a remote in- 
put device for a computer (50). 

35 

12. A system according to daim 11 wherein the first 
processing step comprises converting the select- 
ed input signals into object-level graphics code 
suitable for the computer (50). 

40 



13. A system according to daim 12 wherein the com- 
puter comprises a multi-user computer. 

14. Asystem according to daim 12 or 13 wherein the 
second processing means (3) is adapted to con- 
vert the object-level graphics code into input for 
an apiriication programme (52) running on the 
computer. 

15. A communication system comprising at least one 
mobile handheld telephone handset (1) adapted 
to communicate via a wireless telephony medium 
(3) with a telephone network handling system 
(3.4); 

wherein the handset compnses means 
(10.8.11-13) to receive inputfrom a user and pro- 
duce signals dependent thereupon, means (7) to 
adapt speech input to produce a voice transmis- 



sion signal as part of a telephone conversation 
with a third party; and means (9) to transmit the 
voice transmission signal via the wireless teleph- 
ony medium (3); and 

wherein the telephone network handling 
system (3.4) comprises means to receive the 
voice transmission signal, and means to forward 
the voice signal to a third party (29,30); 

characterised in that the handset is adapt- 
ed to transmit and receive Input/output date via 
the wireless telephony medium and the tele- 
phone network handling system whereby the 
handset acts as an Input/output device for an ap- 
plication program executing on a comjMJter posi- 
tioned renwte from the handset and attached to 
the telephone network handling system. 

16. Asystem according to daim 15, wherein output 
from the remotely executing application Is trans- 
mitted to be displayed on the handset 

17. Asystem according to daim 15 or 16 wherein se- 
lected Input signals from the user are transmitted 
as an output signal to the computer for Input to the 
remotely executing application. 

18. Asystem according to claim 16 or 17 wherein the 
Input and/or output signals are in the form of an 
object-level graphics description protocol. 

19. A system according to any of claims 15 to 18 
wherein the computer executes a plurality of ap- 
plicatton programs each application program be- 
ing accessed by a different remote handset 

20. A system according to any of the preceding 
claims, wherein the telephone network handling 
system comprises a circuit switched system. 

21. A method of telephonic speech recognition in a 
system according to any of daims 1 to 14. where- 
in said first processing step comprises analysing 
speech signals obtained from speech input ana- 
lysing said speech signals so as to identify speech 
elements of. said speech input and produdng date 
representative of said speech elements of said 
speech input and wherein said second processing 
step comprises donduding a speech recognitkm 
procedure on the basis of the speech elements rep- 
resented by sakJ transmitted date. 

22. A method according to daim 21, wherein sakJ 
speech elements comprise phonemes. 

23. A method according to daim 21, wherein said 
analysis of sakJ speech signals is conducted at a 
sut>-phoneme level. 
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1. Claims 1-5,20 (as far as depending on 1-5), 
21-23 (as far as depending on 1-5): 
ST^litting of speech recognition process betv/een 
handset and exchange 

2. Claims 6-10,20 {as far as depending on 6-10), 
21-23 (as far as depending on 6-10); 
Handwriting irecognition split between handset and 
exchange 

3. Claims 11-19,20 (as far as depending on 11-19 )/2i-23 
(as far as depending on 11-14): 

Use of handset as a remote control 
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